08. Constructing Financial State Spaces Part 2

AI For Trading C5 L2 A03 Practical Understanding V2.2

Feature Engineering and State Spaces

This session focuses on preparing data for reinforcement learning by creating feature vectors and feature matrices.

Key Concepts:

  • Feature Vector Construction:

    • Organize feature values into a vector format for machine learning models.
    • Example features: closing price P_t, 5-day simple moving average SMA5_t, and volume V_t.
    • Each feature vector is a snapshot of selected features at a particular time (e.g., t=4, t=5, t=6).

  • State Space and Feature Matrix in Reinforcement Learning:

    • State space involves organizing feature vectors over time to enable temporal understanding.
    • Feature Matrix: Consists of successive feature vectors forming a matrix (or state representation) necessary for reward calculation.
    • Includes present and historical data (e.g., for t=5, include data from t=4 and t=5).

  • Importance of State Representation:

    • Ensures the agent learns effectively from data without excessive dimensionality.
    • Poor state representation can lead to suboptimal learning and decisions.

  • Normalization:

    • Essential for stabilizing learning by ensuring no feature overwhelms others due to its scale.

Future discussion will explain normalization's role in financial markets.

Supplementary Material - Understanding Indicator Combinations

The following Investopedia article is helpful for understanding what your state space may be conveying to your model. This article will help you understand how a human analyst may interpret various of technical indicators. While your model may not arrive at the exact same interpretation, it remains useful for deepening your understanding of how these state spaces work within the financial problem space.

Technical Indicator: Definition, Analyst Uses, Types and Examples

When constructing feature vectors from raw data, what is the typical format for including multiple features?
SOLUTION: An ordered collection of numerical values representing each feature.

How is a feature matrix constructed, and why is it important in reinforcement learning?
SOLUTION: By stacking feature vectors from multiple time steps to capture temporal dependencies.

In the example provided, what does a feature matrix with a window size of m=2 represent?
SOLUTION: The feature vectors for the current time step and the previous time step.

Next Concept